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Claims: 

1. A system for modifying images characterized in that it includes the 
ability to assign colour intensity values to pixels exposed during image 
manipulation operations comprising: 

means for using the color intensity values of remaining pixels in 
the original image, and 

means for assigning color values to the exposed pixels that are 
similar to those of the surrounding pixels so that the exposed 
regions blends smoothly with the surrounding region. 

2. The system as claimed in claim 1, including means for assigning color 
values when pixels are exposed due to manual modifications of the 
image. 

3. The system as claimed in claim 1, including means for assigning color 
values when the pixels are exposed due to automatic modifications of 
said image. 

4. The system as claimed in claim 1, wherein said means for assigning 
colour values assigns the value of colour intensity based on the colour 
intensity value function determined at that pixel's location. 

5. The system as claimed in claim 4, wherein the said color intensity value 
function used to assign values to exposed pixels is determined using the 
image model available a-priori. 

6. The system as claimed in claim 4, wherein said color intensity value 
function used to assign values to exposed pixels is determined by fitting a 
function to the known color intensity values of pixels in the boundary 
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regions of the exposed surrounding pixels. 

The system as claimed in claim 6, wherein the color intensity value 
function is approximated using energy minimization along with boundary 
conditions. 

The system as claimed in claim 6, wherein said color intensity value 
function is based on a multi-point derivative estimation which is used as 
an approximation to the smoothness functional that is used to constrain 
the energy minimization. 

The system as claimed in claim 1 ? wherein said means for assigning 
colour values assigns a colour value to an exposed pixel equal to the 
color intensity value of its mirror-image pixel found by looking into a 
hypothetical mirror placed along the boundary of the exposed region. 

The system as claimed in claim 1, wherein said means for assigning 
colour values segments the original image to restrict the pixels, which are 
used to assign intensity values to the exposed pixels by using: 

means for copying the intensity value from only that pixel location 
that belongs to the same segment as the boundary pixel which 
intersects the normal to the boundary and the boundary, 
means for copying the intensity value from only that pixel location 
that belongs to a segment with a minimum contact length with the 
exposed region. 
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1 1 . The system as claimed in claim 1 , wherein the user explicitly restrains the 
area from where intensity values may be used to determined the intensity 
values that have to be assigned to the exposed pixels. 

5 12. The system as claimed in claim 1, wherein for an original image that has 
been compressed using a lossy compression technique and wherein said 
means for assigning colour values to the exposed pixels includes: 

means for performing median filtering on the boundary pixels not 
belonging to the exposed region but surrounding the exposed 
10 region where the median filtering mask does not use the pixels 

inside the exposed region; 

means for ensuring that the pixels that are closer than a 
predetermined distance to the exposed region boundary are not 
used while determining the intensity values of the exposed pixels. 

15 

13. A method for modifying images characterized in that it includes the 
ability to assign colour intensity values to pixels exposed during image 
manipulation operations comprising: 

using the color intensity values of remaining pixels in the 
20 original image, and 

assigning color values to the exposed pixels that are similar 
to those of the surrounding pixels so that the exposed 
regions blends smoothly with the surrounding region. 
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The method as claimed in claim 13, including assigning color values 
when the pixels are exposed as a result of manual modification of image 
regions by the user. 
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15. The method as claimed in claim 13, including assigning color values 
when the pixels are exposed as a result of automatic modification of said 
image regions. 

5 16. The method as claimed in claim 13, wherein said assigning of colour 
values assigns the value of colour intensity based on the colour intensity 
value function determined at that pixel's location. 

17. The method as claimed in claim 16, wherein said color intensity value 
10 function used to assign values to exposed pixels is determined using the 

image model available a-priori. 

18. The method as claimed in claim 16, wherein said color intensity value 
function used to assign values to exposed pixels is determined by fitting a 

15 function to the known color intensity values of pixels in the boundary 

regions of the exposed pixels. 

19. The method as claimed in claim 18, wherein the color intensity value 
function is approximated using energy minimization along with boundary 

20 conditions. 

20. The method as claimed in claim 18, wherein said color intensity value 
function is based on a multi-point derivative estimation, which is used as 
an approximation to the smoothness functional that is used to constrain 

25 the energy minimization. 

21. The method as claimed in claim 13, wherein said assigning of colour 
values assigns a colour value to an exposed pixel equal to the color 
intensity value of its mirror-image pixel found by looking into a 
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hypothetical mirror placed along the boundary of the exposed region. 

22. The method as claimed in claim 13, wherein said assigning of colour 
values segments the original image to restrict the pixels, which are used 
to assign intensity values to the exposed pixels by using: 

copying the intensity value from only that pixel location that 
belongs to the same segment as the boundary pixel which 
intersects the normal to the boundary and the boundary, 
copying the intensity value from only that pixel location that 
belongs to a segment with a minimum contact length with the 
exposed region. 

23. The method as claimed in claim 13, wherein the user explicitly restrains 
the area from where intensity values may be used to determined the 
intensity values that have to be assigned to the exposed pixels. 

24. The method as claimed in claim 13, wherein for an original image that 
has been compressed using a lossy compression technique and wherein 
said means for assigning colour values to the exposed pixels includes: 

performing median filtering on the boundary pixels not belonging 
to the exposed region but surrounding the exposed region where 
the median filtering mask does not use the pixels inside the 
exposed region; 

ensuring that the pixels that are closer than a predetermined 
distance to the exposed region boundary are not used while 
determining the intensity values of the exposed pixels. 
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25. A computer program product comprising computer readable program 
code stored on computer readable storage medium embodied therein for 
modifying images characterized in that it includes the ability to assign 
colour intensity values to pixels exposed during image manipulation 

5 operations comprising: 

computer readable program code means configured for using the 
color intensity values of remaining pixels in the original image, 
and 

computer readable program code means configured for assigning 
10 color values to the exposed pixels that are similar to those of the 

surrounding pixels so that the exposed regions blends smoothly 
with the surrounding region. 

26. The computer program product as claimed in claim 25, including 
15 computer readable program code means configured for assigning color 

values to pixels exposed as a result of manual modifications of image 
regions. 

27. The computer program product as claimed in claim 25, including 
20 computer readable program code means configured for assigning color 

values to pixels exposed as a result of automatic modifications of said 
image regions. 

28. The computer program product as claimed in claim 25, wherein said 
25 computer readable program code means configured for assigning colour 

values assigns the value of colour intensity based on the colour intensity 
value function determined at that pixel's location. 
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29. The computer program product as claimed in claim 28, wherein said 
color intensity value function used to assign values to exposed pixels is 
determined using the image model available a-priori. 

30. The computer program product as claimed in claim 28, wherein said 
color intensity value function used to assign values to exposed pixels is 
determined by fitting a function to the known color intensity values of 
pixels in the boundary regions of exposed pixels. 

3 1 . The computer program product as claimed in claim 30, wherein the color 
intensity value function is approximated using energy minimization along 
with boundary conditions. 

32. The computer program product as claimed in claim 30, wherein said 
color intensity value function is based on a multi-point derivative 
estimation which is used as an approximation to the smoothness 
functional that is used to constrain the energy minimization. 

33. The computer program product as claimed in claim 25, wherein said 
computer readable program code means configured for assigning colour 
values assigns a colour value to an exposed pixel equal to the color 
intensity value of its mirror-image pixel found by looking into a 
hypothetical mirror placed along the boundary of the exposed region. 

34. The computer program product as claimed in claim 25, wherein said 
computer readable program code means configured for assigning colour 
values segments the original image to restrict the pixels, which are used 
to assign intensity values to the exposed pixels by using: 
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computer readable program code means configured for copying 
the intensity value from only that pixel location that belongs to the 
same segment as the boundary pixel which intersects the normal to 
the boundary and the boundary, 

computer readable program code means configured for copying 
the intensity value from only that pixel location that belongs to a 
segment with a minimum contact length with the exposed region. 

35. The computer program product as claimed in claim 25, wherein the user 
explicitly restrains the area from where intensity values may be used to 
determined the intensity values that have to be assigned to the exposed 
pixels. 

36. The computer program product as claimed in claim 25, wherein for an 
original image that has been compressed using a lossy compression 
technique and wherein said means for assigning colour values to the 
exposed pixels includes: 

computer readable program code means configured for performing 
median filtering on the boundary pixels not belonging to the 
exposed region but surrounding the exposed region where the 
median filtering mask does not use the pixels inside the exposed 
region; 

computer readable program code means configured for ensuring 
that the pixels that are closer than a predetermined distance to the 
exposed region boundary are not used while determining the 
intensity values of the exposed pixels. 
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